Distribution-free robust linear regression

We study random design linear regression with no assumptions on the distribution of covariates and a heavy-tailed response variable. In this distribution-free setting, we show that boundedness conditional second moment given is necessary sufficient condition for achieving nontrivial guarantees. As starting point, prove an optimal version classical in-expectation bound truncated least squares estimator due to Gy\"{o}rfi, Kohler, Krzy\.{z}ak, Walk. However, procedure fails constant probability some distributions despite its performance. Then, combining ideas squares, median-of-means procedures, aggregation theory, construct non-linear excess risk order $d/n$ sub-exponential tail. While existing approaches focus proper estimators return functions, highlight improperness our attaining guarantees in setting.